1. Field of the Invention
The present invention generally relates to methods of image feature coding and image search and, more particularly, to a method of image feature coding and a method of image search in which features of analog or digital data of moving images or still images are extracted and coded so that the image data is searched using the coded feature.
2. Description of the Related Art
FIG. 1 is a block diagram showing a related-art image search system described in Denshi Jouhou Tsuushin Gakkai Rombunshi D-II Vol. 79-D-II No. 4, April, 1996, pp.476-483. Referring to FIG. 1, the system comprises a pre-processing unit 91 for segmenting a still image 201, attaching a keyword to each of the resulting segments and outputting a conceptual keyword 203 and a scene descriptive keyword 204; a search tool 92 for receiving a keyword 205 from a user 97 and retrieving the still image 201 by checking the input keyword against the conceptual keyword 203 and the scene descriptive keyword 204.
The conceptual keyword 203 is a keyword indicating color information and feature of a segment. The scene keyword 204 is a keyword representing a feature of the segment using descriptive words relating to position, color, shape, size and orientation.
The pre-processing unit 91 of FIG. 1 comprises a segmentation unit 93 for segmentation of the still image 201; a conceptual keyword extraction unit 94 for extracting the conceptual keyword 203 from the color and feature of the segment produced by the segmentation unit 93; and a scene descriptive keyword providing unit 95 for assigning the scene keyword 204 to the image feature of the segment produced by the segmentation unit 93, by receiving a predicate description 202 selected by a user 96 from a set of predefined predicates.
The search tool 92 of FIG. 1 comprises a feature identifying unit 98 for checking the keyword 205 selected by the user 97 against the conceptual keyword 203 and the scene descriptive keyword 204 from the pre-processing unit 91.
A description will now be given of the operation.
When the still image 201 is supplied to the pre-processing unit 91, the segmentation unit 93 segments the still image 201. The conceptual keyword extracting unit 94 extracts the conceptual keyword 203 from the color and feature of the segment. More specifically, the conceptual keyword extracting unit 94 starts with a conceptual keyword associated with the color information to arrive at the conceptual keyword 203.
The scene descriptive keyword providing unit 95 provides the scene keyword 204 to the image feature of the segment, by receiving the predicate description 202 from the user 96.
When searching for the still image 201, the user 97 inputs the keyword 205, selected from a prepared set of conceptual keywords 203 and scene keywords 204, to the search tool 92. The feature identifying unit 98 retrieves the still image 201 requested by the-user 97, based on the keyword 205 provided by the user 97, the conceptual keyword 203 and the scene descriptive keyword 204 from the preprocessing unit 91.
Since the target of the image search system described above is the still image 201, there is a drawback in that it is difficult to search for moving images.
In addition, since not much consideration is given to how the keywords are provided and stored, a one-to-one correspondence between an image server and a client (search tool 92) is a prerequisite. Therefore, according to the related art, an image search system where a large number of users are capable of searching for images using a variety of search tools via a network cannot be built.
Accordingly, a general object of the present invention is to provide a method of image feature coding and a method of image search in which the aforementioned drawbacks are eliminated.
Another and more specific object is to provide a method of image feature coding and a method of image search in which a large number of users can search for images using a variety of search tools.
The aforementioned objects can be achieved by an image feature coding method comprising the steps of: extracting segments of image areas from an image frame; attaching a segment number to each of the extracted segments; assigning a representative color to each of the extracted segments; computing a relative area of each of the segments with respect to the image frame; coding the representative color and the relative area to produce a feature of the image; and generating a feature stream corresponding to the image having the feature encoded therein.
The segments may be extracted from the image frame in accordance with color information, and the color information used in extracting the segments is assigned to the extracted segments as the representative color.
The segments from adjacent image frames may be checked for identity match, and those segments determined to match each other are given a same segment number.
The segments may be tracked from image frame to image frame so as to determine movement information relating to the segments that match each other.in identity, the movement information is generated by coding to produce the feature of the segments, and the feature stream, having the feature thus produced encoded therein, is generated.
An appropriate key frame that provides a key for a search is extracted from a group of image frames of a video signal, whereupon the segments are extracted from the extracted key frames.
A reduced image of the key frame may be generated by averaging pixels located in respective areas of the key frame, the reduced image is coded to produce the feature of the key frame, and the feature stream, having the feature thus produced encoded therein, is generated.
The aforementioned objects can also be achieved by an image searching method using first storage unit for storing image frames and a second storage unit for storing a feature stream having features of the image frames encoded therein, comprising the steps of: decoding the features stored in the second storage unit, in accordance with a search instruction from a user; and checking the decoded features against a search criteria provided by the user for identity match.
The features stored in the second storage unit may include a representative color of a segment constituting an area in the image frame, and the search criteria from the user may include the representative color.
The features stored in the second storage unit may include a relative area of a segment, constituting an area in the image frame, with respect to the image frame, and the search criteria from the user may include the relative area.
The features stored in the second storage unit may include movement information related movement between adjacent image frames, and the search criteria from the user may include the movement information.
The features stored in the second storage unit may include a reduced image of the image frame, the decoded feature may be checked against the search criteria from the user, and the reduced image may be presented to the user.
The features stored in the second storage unit may include information indicating whether a designated object is captured in the image frame.
The features stored in the second storage unit may include information indicating whether a designated object is captured subsequent image frames.
The features stored in the second storage unit may include information indicating whether a designated object is captured previous image frames.
Priority given to the decoded feature when checking the decoded feature against the search criteria from the user may be presented to the user.
A plurality of decoded features may be checked from an viewpoint against a plurality of search criteria from the user for a match from an overall perspective.